Autofocus has become a standard feature on both film and digital cameras. Despite years of innovation and development, autofocus still has several problems that often result in out-of-focus images.
The biggest issue is user intent. In a scene with objects at multiple distances, the camera must choose which part of the image the user would like to be in focus. The conventional solution is to focus on center region of the image, assuming the subject is in the center. This technique fails in the classic case of two people standing side-by-side, with the camera focusing on the background between them.
In addition, conventional autofocus techniques on digital cameras are also slow, power-hungry, and inaccurate in low contrast or low light conditions.
One sophisticated approach to the problem of user intent for autofocus is the use of eye tracking. As a user looks into the viewfinder, an IR emitter and detector array track where the user is looking while framing the shot. The camera then uses an autofocus zone or spot at that position in the scene. An autofocus algorithm then proceeds conventionally, using the image as seen through the camera lens. However, this solution still suffers from the speed, power, and accuracy issues of conventional autofocus.
Prior art relating to the present invention is as follows.
U.S. Pat. No. 6,072,443 discloses an “ocular projection display (12) projects an image directly to the eye (26) of the user (10). The focus of the image may be varied to allow for different user profiles or to relieve the stress of maintaining a fixed focus over a prolonged period of time. Optionally, the ocular projection display (12) can include a location and distance sensor (46) for identifying the location of the user's eyes for proper aiming of the image to the eyes of the user and focus detection circuitry (54) to correct for the user's focusing abilities.”
U.S. Pat. No. 5,857,120 discloses an “eye axis detector for detecting to which position the eye axis of a user is directed in a picture, comprising means for dividing and specifying the picture into a plurality of pictures, and means for extracting the watching point of the user using a plurality of types of eye axis information concerning the eye axis in each area within such divided picture.”
U.S. Pat. No. 5,155,516 discloses a “view finder optical system including a window of an eye piece portion comprising an eye cup having a contact surface, a shape of the contact surface being made so as to incline an axis of the lens with respect to an axis of the eye piece. The shape of the contact surface is curved corresponding to the lens's shape. The contact surface is made so as to conform to a lens of glasses.”
U.S. Pat. No. 5,608,489 discloses an “anamorphic lens system in an eye direction detection device, applied to an observing optical system, modifies illumination light from a light source to be parallel in a lateral direction of a user's eye positioned to look through the observing optical system, dispersed in a vertical direction of the position of the user's eye, and substantially centered on the position of the user's eye. The anamorphic lens system transmits the illumination light from the light source to an illumination light exit surface positioned away from the optical axis of the observing optical system, and the illumination light is projected toward the position of the user's eye from the illumination light exit surface to separate signals from the image of the eye and an image formed by eyeglasses, if worn. The illumination light exit surface is a rectangular window having a longitudinal side parallel to longitudinal side of a field of view of the observing optical system, and is positioned below the position of a user's eye.”
U.S. Reissue Pat. No. RE36,237 (which corresponds to U.S. Pat. No. 5,155,516) discloses a “view finder optical system including a window of an eye piece portion comprising an eye cup having a contact surface, a shape of the contact surface being made so as to incline an axis of the lens with respect to an axis of the eye piece. The shape of the contact surface is curved corresponding to the lens's shape. The contact surface is made so as to conform to a lens of glasses.”
U.S. Pat. Nos. 5,327,191 and 5,557,364 disclose an “eye direction detecting apparatus for a camera having a light transferring system for guiding a beam of parallel light rays to an eye of a photographer includes a light receiving system having a light receiving portion on which a first Purkinje image based on specular reflection of a cornea of the eye and reflecting light from a retina of the eye is formed, the light receiving portion generating a light receiving output. The apparatus further includes a processing circuit for detecting the eye direction of the eye based on the light receiving output of the light receiving portion. Further, according to the teachings of the present invention, including an optical member having certain identically inclined surfaces prevents refracted light from forming a ghost image within the light receiving system of an eye direction detecting apparatus.”
U.S. Pat. No. 5,331,149 discloses an “eye tracking system is disclosed which is comprised of an eye tracking module formed of a display joined to a photodetector array. Each pixel in the display is aligned with a corresponding photodetector. An image generated by the display is projected onto a viewing screen or toward a viewer. Axial light rays from the display pixels are reflected by the eye and detected by a respective photodetector which generates an electrical signal indicative of eye position.”
U.S. Pat. No. 5,614,985 discloses a “camera having sight line detection means for detecting the line of sight of an observer, focus detection means having plural focus detection areas and capable of focus detection in each focus detecting area, and selection means for selecting at least one of the plural focus detecting areas, based on the information of the line of sight detected by the sight line detection means, wherein the selection means is adapted to select the focus detecting area based on the result of focus detection by the focus detection means, according to the state of detection of the line of sight by the sight line detection means.”
U.S. Pat. No. 5,882,301 discloses that “Relative directions of excitation and photoreceiving optical systems are so set that an angle formed by optical axes thereof in the air is 14°, and an eyeball is fixed in such a direction that its ocular axis divides the angle formed by the optical axes into two equal parts. On a light incidence side of a one-dimensional solid-state image pickup device of the photoreceiving optical system, a slit is arranged for inputting measuring light components generated from portions of the eyeball having different depth positions on an excitation light beam in photoelectric conversion elements of different positions of the image pickup device. The measuring light components generated from the respective portions of the eyeball parallel to the optical axis are incident upon the one-dimensional solid-state image pickup device through the slit and simultaneously detected, so that the positions of the photoelectric conversion elements and measuring light component generating positions at the eyeball correspond to each other.”
U.S. Pat. No. 5,610,681 discloses an “apparatus having an irradiation device for irradiating the eye of an observer; a sensor having a number of pixels with a set pitch; an image forming optical unit for imaging light reflected by the eye onto the sensor; and an electronic circuit for making a signal denoting the direction of the line of sight of the eye in accordance with an output from the sensor, wherein the relationship expressed by Pitch X/β<0.41 mm is satisfied when an assumption is made that the image forming magnification of the image forming unit is β and the pitch of the pixels of the sensor is Pitch X so that accuracy in detecting the line of sight of the eye is improved.”
U.S. Pat. No. 6,014,524 discloses a “camera comprising: finder means for inspecting an object, illumination means for illuminating an operator's eye by which the operator looks in at the finder means, a condensing optical system for condensing a reflected light from the operator's eye, photoelectric converter means for receiving the condensed reflected light, calculation means for calculating a direction of a visual axis of the operator's eye on the basis of an output of the photoelectric converter means, and condition setting means operable in response to the result of the calculation of the calculation means, for setting taking conditions of the camera”
U.S. Pat. No. 6,394,602 discloses an “optical instrument, such as a microscope or a camera, is provided for forming a viewable image of an object. The optical instrument comprises an objective lens for forming the viewable image at an image plane, an eye sensor for sensing the direction of gaze of a user viewing the viewable image and means for controlling a controllable function of the optical instrument in dependence upon the sensed direction of gaze. The eye sensor comprises a sensor lens for focusing light reflected from the retina of the user, an imaging transducer located at a plane which is conjugate to the image plane with respect to the sensor lens, for generating an electrical image sign of a portion of the user's retina, a memory for storing retinal image information of the user and circuitry for comparing the retinal image signal generated by the imaging transducer with the stored retinal image information to generate gaze information indicative of the direction of gaze of the user”
U.S. Pat. No. 6,027,216 discloses “Apparatus and method are provided for assessing the direction of fixation of an eye by detecting polarization-related changes in light retroreflected from the fundus of the eye. Nerve fibers in the retina of the eye are birefringent and alter the polarization state of light traversing them as a function of their orientation. The nerve fibers are arrayed in a characteristic pattern in the retina, specifically radiating outward from the fovea and converging to the optic nerve head. By assessment of polarization-related changes in retroreflected light from multiple retinal areas either sequentially or simultaneously, characteristic birefringence signatures of portions of the retina can be identified which are used to assess the direction of fixation of the eye. In addition, interference from the corneal birefringence is reduced by using incident light having a polarization state that is substantially independent of meridional direction. Circularly polarized light or non-polarized light is used for the assessment. Interference from the corneal birefringence is reduced still further by detecting polarization-related changes that are substantially independent of the meridional direction of the corneal birefringence. This is accomplished by detecting changes in ellipticity by measuring solely the Stokes parameter S3 or by measuring any two Stokes parameters. An alternative is measuring the overall intensity of the retroreflected light when the dichroism of the lutein pigment particles in the vicinity of the fovea is used for the assessment.”
However, none of the prior art references discloses or suggests using retroreflected eye focus measurements to measure the focus distance of the eye, and use this distance to set the focus of the camera.
It is an objective of the present invention to provide for a digital camera, viewfinder apparatus and methods that use retroreflected eye focus measurements to provide autofocus, and that improves over conventional implementations.